Method and apparatus for detecting scans in real-time

ABSTRACT

A method and apparatus for detecting scans are described. In one example, a plurality of flows is allocated into a plurality of bins associated with different source internet protocol (SIP) addresses. A set of bin characteristics for at least one bin of the plurality of bins is generated if the at least one bin reaches a predefined flow capacity. Afterwards, the set of bin characteristics is compared to a scan characteristics list to determine if a potential scan exists.

This application is a continuation of U.S. patent application Ser. No.13/088,230, filed Apr. 15, 2011, which is currently allowed and is acontinuation of U.S. patent application Ser. No. 11/321,169, filed Dec.29, 2005, now U.S. Pat. No. 7,930,748, both of which are hereinincorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Embodiments of the present invention generally relate totelecommunications systems and, more particularly, to a method andapparatus for detecting scans in a stream of data packets over anetwork.

2. Description of the Related Art

Reconnaissance or scanning typically serves as an initial indication ofnetwork intrusion. Whether scanning is conducted automatically by a wormor manually by a hacker, the ultimate goal is to gather informationregarding the vulnerabilities of the network or associated machines.Thus, although not harmful in itself, a scan may lead to moredestructive attacks or be indicative of potentially dangerous activity.Consequently, the detection of scans may serve as an effective methodfor early detection of various attacks (e.g., worms) or potentialattacks (e.g., BotNets).

Thus, there is a need in the art for a method and apparatus fordetecting scans.

SUMMARY OF THE INVENTION

In one embodiment, a method and apparatus for detecting scans aredescribed. Specifically, a plurality of flows is allocated into aplurality of bins associated with different source internet protocol(SIP) addresses. A set of bin characteristics for at least one bin ofthe plurality of bins is generated if the at least one bin reaches apredefined flow capacity. Afterwards, the set of bin characteristics iscompared to a scan characteristics list to determine if a potential scanexists.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the presentinvention can be understood in detail, a more particular description ofthe invention, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1 is a block diagram depicting an exemplary embodiment of acommunication system in accordance with the invention;

FIG. 2 is a flow diagram depicting an exemplary embodiment of a methodfor detecting scans in accordance with one or more aspects of theinvention;

FIG. 3 is a flow diagram depicting an exemplary embodiment of a methodfor detecting multiport scans in accordance with one or more aspects ofthe invention; and

FIG. 4 is a block diagram depicting an exemplary embodiment of acomputer suitable for implementing the processes and methods describedherein.

DETAILED DESCRIPTION

To better understand the present invention, FIG. 1 illustratescommunication architecture 100 comprising an example network, e.g., apacket network related to the present invention. Broadly defined, apacket network is a network that is capable of carrying information aspacketized data over an IP network. Exemplary packet networks utilizedby the present invention include Internet protocol (IP) networks, suchas an IPv4 network, an IPv6 network, and the like. An IP network isbroadly defined as a network that uses Internet Protocol to exchangedata packets. Furthermore, the present invention should not beinterpreted to be limited by this particular illustrative architectureor limited to this type of network. For example, the present inventionmay be utilized to detect scans in a stream of data packets over anInternet service provider (ISP) network, a university network, or even asingle home computer.

FIG. 1 is a block diagram depicting an exemplary configuration of acommunication system 100 constructed in accordance with one or moreaspects of the invention. A plurality of endpoint devices 102-104 areconfigured for communication with the core packet network 110 via anaccess network 101. Similarly, a plurality of endpoint devices 105-107are configured for communication with the core packet network 110 (e.g.,an IP based core backbone network supported by a service provider) viaan access network 108. The network elements 109 and 111 may serve asgateway servers or edge routers for the network 110. Those skilled inthe art will realize that although only six endpoint devices, two accessnetworks, and five network elements (NEs) are depicted in FIG. 1, thecommunication system 100 may be expanded by including additionalendpoint devices, access networks, and border elements without alteringthe present invention.

The endpoint devices 102-107 may comprise customer endpoint devices suchas personal computers, laptop computers, personal digital assistants(PDAs), servers, and the like. The access networks 101 and 108 serve asa means to establish a connection between the endpoint devices 102-107and the NEs 109 and 111 of the core network 110. The access networks101, 108 may each comprise a digital subscriber line (DSL) network, abroadband cable access network, a local area network (LAN), a wirelessaccess network (WAN), and the like. Some NEs (e.g., NEs 109 and 111)reside at the edge of the core infrastructure and interface withcustomer endpoints over various types of access networks. An NE istypically implemented as an edge router, a media gateway, a borderelement, a firewall, and the like. An NE may also include a componentthat resides within the network (e.g., NEs 118-120) such as a honeypot,a tarpit, a mail server, or like device. Similarly, an NE is responsiblefor providing flow data or flows to an application server 112. A flow(or flow data) comprises a set of packets wherein each packet of theflow is characterized by the same source IP (SIP) address, destinationIP (DIP) address, source portal (sport), destination portal (dport), andprotocol. A flow may also be defined by a FIN flag packet that indicatesthe end of the flow.

The core network 110 also comprises an application server 112 thatcontains a scan detection module (SDM) 114, scan rules list (SRL) 113,and a database 115. The application server 112 may comprise any serveror computer that is well known in the art (also see FIG. 4). In oneembodiment of the present invention, the server 112 comprises adedicated application server that is configured to receive and processflow data from the network NEs in order to detect scans using the SDM114. The database 115 may be any type of electronic collection of datathat is well known in the art. The database 115 is used by the server112 to store detected scans in a list of scans 116.

In one embodiment, the server 112 also contains a scan rules list (SRL)113 that comprises a list of predefined scan classifications or scantypes. The SRL 113 is used by the server 112 to classify the flow datain accordance to a scan type. For example, the scan type may becategorized as an A-class scan, an A-class random scan, a B-class scan,a B-class random scan, a C-class scan, a port scan, a completely randomscan, and the like. These scan types may also include Internet controlmessage protocol (ICMP) scans. Notably, each scan type classification ischaracterized by different characteristics or properties that may becategorized as being constant (x), distributed (*), unknown (?), or anyof the previous three (“any”). For example, a C-class scan ischaracterized by (i) having a bytes per packet ratio that is constant(e.g., BPR=x), (ii) having a constant destination IP address for thefirst three octets and having a distributed fourth octet (e.g.,DIP=x.x.x.*), (iii) originating from any source port (e.g., sport=any),(iv) having a constant destination port (e.g., dport=x), and (v) havinga common protocol (e.g., protocol=x).

The characteristics of other non-icmp scan types include: an A-classscan (DIP=x.*.x.x, sport=any, dport=x, protocol=x, BPR=x), an A-classrandom scan (DIP=x.*.*.*, sport=any, dport=x, protocol=x, BPR=x), aB-class scan (DIP=x.x.*.x, sport=any, dport=x, protocol=x, BPR=x), aB-class random scan (DIP=x.x.*.*, sport=any, dport=x, protocol=x,BPR=x), a general random scan (DIP=*.*.*.*, sport=any, dport=x,protocol=x, BPR=x), and a port scan (DIP=x.x.x.x, sport=any, dport=*,protocol=x, BPR=x). Similarly, the characteristics of icmp scan typesinclude: an A-class icmp scan (DIP=x.*.x.x, sport=any, dport=any,protocol=1, BPR=x, icmp type=x), an A-class random icmp scan(DIP=x.*.*.*, sport=any, dport=any, protocol=1, BPR=x, icmp type=x), aB-class icmp scan (DIP=x.x.*.x, sport=any, dport=any, protocol=1, BPR=x,icmp type=x), a B-class random icmp scan (DIP=x.x.*.*, sport=any,dport=any, protocol=1, BPR=x, icmp type=x), a general random icmp scan(DIP=*.*.*.*, sport=any, dport=any, protocol=1, BPR=x, icmp type=x), anda C-class icmp scan (DIP=x.x.x.*, sport=any, dport=*, protocol=1, BPR=x,icmp type=x).

FIG. 2 is a flow diagram depicting an exemplary embodiment of a method200 for detecting scans as related to one or more aspects of theinvention. Although the present invention is described as utilizingflow-level data, the method 200 can also be used with packet-level dataas well. The method 200 begins at step 202 and proceeds to step 204where a plurality of flows (e.g., metadata) is received. In oneembodiment, a NE of the core network reads in the flows and provides theflow data to the application server 112.

At step 206, each separate flow is recorded in one of a plurality ofbins, e.g., in accordance to the source IP (SIP) addresses associatedwith the received flows. In one embodiment, the applications server 112establishes a plurality of bins that is associated with a plurality ofdifferent SIP addresses. As separate flow data is received by the NE,the flows are allocated into the appropriate bin depending on the origin(i.e., the SIP address) of the flow(s).

At step 208, a determination of whether at least one bin has beenfilled. In one embodiment, a bin is filled when the number of flowsreaches a predefined capacity value, MAXBIN (e.g., MAXBIN=15). If noneof the bins have been filled, then the method 200 loops back to step 206where the flows continue to be allocated into the bins. If at least onebin is filled (e.g., 15 flows have been collected), then the method 200continues to step 210, where bin characteristics for the collective flowdata (e.g., 20 flows) in the filled bin are generated. In oneembodiment, the application server 112 generates the bin characteristicsusing the SDM 114.

In one embodiment, the bin characteristics include the source IP (SIP)address, destination IP (DIP) address, destination port (dport) (1^(st),2^(nd), 3^(rd), and 4^(th) octets), source port (sport), protocol,transmission control protocol (TCP) flags, flow count, bytes per packetratio (BPR), and icmp type (if applicable). The summary characteristicsalso contain additional information for meeting threshold requirement orfor outputting information concerning the scans such as, starting andending timestamps, packet count, and byte count. Notably, each bincharacteristic is then categorized as being constant (“x”), distributed(“*”), or unknown (“?”). For example, if most of the flows in the binare directed to a common destination port number, the destination portcharacteristic would be classified as being constant (i.e., dport=“x”).Conversely, if most of the flows in the bin have different destinationport numbers, then the characteristic would be categorized asdistributed (i.e., dport=“*”). Depending on the embodiment, “most” maybe defined by a predefined threshold, BINTHRESH (e.g., BINTHRESH=13),which represents a number that must be met or exceeded to properlyclassify a bin characteristic. For example, if BINTHRESH=13, then 13 ofthe 15 flows contained in the bin must demonstrate a particular qualityin order to be classified (e.g., if 13 flows all have the same DIPaddress, then the DIP characteristic of the bin is classified as“constant”). Each of the characteristics are processed and categorizedin this manner.

At step 212, the generated bin characteristics are compared to a ruleslist. Specifically, a determination of whether the bin characteristicsmatch any of the entries in a scan rules list 113 is made. If no matchesare found, then the method proceeds to step 222 where the previouslystored bin characteristics are erased (e.g., overwritten by NULL). If amatch is found, then the method continues to step 218.

At step 218, a determination of whether the bin characteristics matchthe previously stored bin characteristics is made. In one embodiment,the server 112 determines if the new bin characteristics match thesummary characteristics of the filled bin. Namely, each bin keeps asummary of previous bin characteristics (e.g., a flow count of aparticular scan class type). If the bin characteristics match thepreviously stored summary characteristics (i.e., the new bincharacteristics are associated with the same scan type, SIP, DIP octetpattern, BPR, etc., as the previously stored summary characteristics.For example, both characteristics correspond to the B-class type scanand have the same SIP, BPR, dport, protocol, and the same 1^(st), 2^(nd)and 4^(th) DIP octets) then the method 200 proceeds to step 220 wherethe flow count is updated. The method 200 then loops back the step 214where the bin is emptied (but the summary characteristics are kept). Ifthe bin characteristics do not match the previously stored summarycharacteristics, then the method 200 proceeds to step 222 where thepreviously stored summary characteristics are overwritten.

At step 224, a determination of whether the previous storedcharacteristics were associated with a flow count that exceeded apredetermined threshold. If the flow count exceeded a threshold (e.g.,MINSCANLENGTH=75), then the method 200 proceeds to step 226 where theflow data associated with the previously stored bin characteristics areidentified as a scan and the characteristics are recorded in the list ofscans. In one embodiment, the characteristics (e.g., the time, duration,the number of flows, the number of packets, the number of bytes, and theranges of each characteristic) are recorded in a list of scans 116located in the database 115. In an alternative embodiment, thedetermination of whether the predefined threshold has been exceeded maybe made after step 220. Namely, the flow data is classified as a scan assoon as the threshold is exceeded (e.g., as soon as 75 flow count isreached), i.e., in real time. The method 200 then loops back to step 214where the bin is emptied and proceeds to step 206 until another bin hasbeen filled. If the previously stored flow count does not exceed thepredefined threshold, then the method 200 loops back to step 214 wherethe bin is emptied and proceeds to step 206 until another bin has beenfilled.

In another embodiment, the present invention may be configured to detectmultiport scans. Multiport scans send packets to several different ports(possibly with different protocols and number of bytes per packet) atthe same DIP address before proceeding to the next IP address. Thesescans may potentially be difficult to detect using the method 200described above since many DIP addresses may be repeated.

FIG. 3 is a flow diagram depicting an exemplary embodiment of a method300 for detecting multiport scans as related to one or more aspects ofthe invention. The method 300 begins at step 302 and proceeds to step304 where a plurality of flows is received. In one embodiment, the NEsof the core network 110 read incoming metadata and provide the flow datato the application server 112.

At step 306, each separate flow is recorded in one of a plurality ofbins in accordance to the flows' respective source IP (SIP) addresses.In one embodiment, the application server 112 establishes a plurality ofbins that is associated with a plurality of different SIP addresses. Asseparate flow data is received by the NE, the flows are allocated intothe appropriate bin depending on the origin (i.e., the SIP address) ofthe flow data.

At step 308, a determination of whether at least one bin has beenfilled. In one embodiment, a bin is filled when the number of flows in agiven bin reaches a predefined value, MAXBIN (e.g., MAXBIN=27). If noneof the bins have been filled, then the method 300 loops back to step 306where the flows continue to be allocated into the bins. If at least onebin is filled (e.g., 27 flows have been collected), then the method 300continues to step 310, where the flows are separated into differentcategories. In one embodiment, the flows are separated into categoriesby like SIP address, destination port, and protocol.

At step 312, the number of unique DIP addresses for a particular BPR(i.e., D(BPR)) and the total number of flows (i.e., “F”) are determinedfor each of the separate categories of step 310.

At step 314, a determination as to whether D(BPR)/F is greater than aminimum ratio (e.g., D(BPR)/F>MINRATIO, where default MINRATIO=0.9) aswell as if D(BPR) is greater than a predetermined threshold representinga number of DIP addresses with a particular BPR (e.g.,D(BPR)>BINTHRESHOLD, where default BINTHRESHOLD=8) is made. If theseconditions are not met, the method 300 continues to step 324 where thepreviously stored bin characteristics are erased (e.g., overwritten byNULL). In one embodiment, the SDM 114 is used to perform thesecalculations. If the conditions are met the method 300 proceeds to step320.

At step 320, a determination of whether the bin characteristics matchthe previously stored summary characteristics is made. In oneembodiment, the server 112 determines if the new bin characteristicsmatch the summary characteristics of the filled bin. If the bincharacteristics match the previously stored summary characteristics thenthe method 300 proceeds to step 322 where the flow count is updated. Themethod 300 then loops back the step 316 where the bin is emptied (butthe summary characteristics are kept).

If the bin characteristics do not match the previously stored summarycharacteristics, then the method 300 proceeds to step 324 where thepreviously stored summary characteristics are overwritten.

At step 326, a determination of whether the previous storedcharacteristics were associated with a flow count that exceeded apredetermined threshold. If the flow count exceeded a threshold (e.g.,MINSCANLENGTH=75), then the method 300 proceeds to step 328 where theflow data associated with the previously stored bin characteristics isidentified as a scan and the characteristics of the scan are recorded inthe list of scans. In one embodiment, these scans are recorded in a listof scans 116 located in the database 115. In an alternative embodiment,the determination of the predefined threshold is exceeded may be madeafter step 322. Namely, the flow data is classified as a scan as soon asthe threshold is exceeded (e.g., as soon as 75 flows are reached).

The method 300 then loops back to step 316 where the bin is emptied andproceeds to step 306 until another bin has been filled. If thepreviously stored flow count does not exceed the predefined threshold,then the method 300 loops back to step 316 where the bin is emptied andproceeds to step 306 until another bin has been filled.

FIG. 4 depicts a high level block diagram of a general purpose computersuitable for use in performing the functions described herein. Asdepicted in FIG. 4, the system 400 comprises a processor element 402(e.g., a CPU), a memory 404, e.g., random access memory (RAM) and/orread only memory (ROM), a module 405 for detecting scans, and variousinput/output devices 406 (e.g., storage devices, including but notlimited to, a tape drive, a floppy drive, a hard disk drive or a compactdisk drive, a receiver, a transmitter, a speaker, a display, a speechsynthesizer, an output port, and a user input device (such as akeyboard, a keypad, a mouse, and the like)).

It should be noted that the present invention can be implemented insoftware and/or in a combination of software and hardware, e.g., usingapplication specific integrated circuits (ASICs), a general purposecomputer or any other hardware equivalents. In one embodiment, thepresent module or process 405 for detecting scans can be loaded intomemory 404 and executed by processor 402 to implement the functions asdiscussed above. As such, the present process 405 for detecting scans(including associated data structures) of the present invention can bestored on a computer readable medium or carrier, e.g., RAM memory,magnetic or optical drive or diskette and the like.

While various embodiments have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. Thus, the breadth and scope of a preferred embodiment shouldnot be limited by any of the above-described exemplary embodiments, butshould be defined only in accordance with the following claims and theirequivalents.

The invention claimed is:
 1. A method for detecting a multiport scan,comprising: allocating, by a processor, a plurality of flows into aplurality of bins based upon a source internet protocol address of eachof the plurality of flows, where each bin of the plurality of bins isassociated with a different source internet protocol address;generating, by the processor, a set of bin characteristics for one binof the plurality of bins when the one bin reaches a predefined flowcapacity; separating, by the processor, a plurality of flows that hasbeen allocated into the one bin into a plurality of categories;determining, by the processor, a number of unique destination internetprotocol addresses and a total number of flows for each category of theplurality of categories; and using, by the processor, the number ofunique destination internet protocol addresses and the total number offlows for each category of the plurality of categories to determine ifthe multiport scan exists.
 2. The method of claim 1, wherein the usingcomprises: determining whether a ratio of the number of uniquedestination internet protocol addresses and the total number of flows isgreater than a predefined ratio value, and whether the number of uniquedestination internet protocol addresses is greater than a predefineddestination internet protocol address value.
 3. The method of claim 2,wherein the using further comprises: when the ratio of the number ofunique destination internet protocol addresses and the total number offlows is not greater than the predefined ratio value, and when thenumber of unique destination internet protocol addresses is not greaterthan the predefined destination internet protocol address value, storedbin characteristics of the one bin are erased.
 4. The method of claim 2,wherein the using further comprises: when the ratio of the number ofunique destination internet protocol addresses and the total number offlows is greater than the predefined ratio value, and when the number ofunique destination internet protocol addresses is greater than thepredefined destination internet protocol address value, a determinationis made as to whether stored bin characteristics of the one bin matchcurrent bin characteristics of the one bin.
 5. The method of claim 4,wherein the using further comprises: when the determination that thestored bin characteristics of the one bin does match the current bincharacteristics of the one bin, a flow count for the one bin is updated.6. The method of claim 4, wherein the using further comprises: when thedetermination that the stored bin characteristics of the one bin doesnot match the current bin characteristics of the one bin, stored bincharacteristics of the one bin are erased.
 7. The method of claim 6,wherein the using further comprises: determining that flow dataassociated with the stored bin characteristics are associated with themultiport scan when the stored bin characteristics of the one bin isassociated with a flow count that exceeded a predefined threshold.
 8. Anon-transitory computer-readable medium storing instructions which, whenexecuted by a processor, cause the processor to perform operations fordetecting a multiport scan, the operations comprising: allocating aplurality of flows into a plurality of bins based upon a source internetprotocol address of each of the plurality of flows, where each bin ofthe plurality of bins is associated with a different source internetprotocol address; generating a set of bin characteristics for one bin ofthe plurality of bins when the one bin reaches a predefined flowcapacity; separating a plurality of flows that has been allocated intothe one bin into a plurality of categories; determining a number ofunique destination internet protocol addresses and a total number offlows for each category of the plurality of categories; and using thenumber of unique destination internet protocol addresses and the totalnumber of flows for each category of the plurality of categories todetermine if the multiport scan exists.
 9. The non-transitorycomputer-readable medium of claim 8, wherein the using comprises:determining whether a ratio of the number of unique destination internetprotocol addresses and the total number of flows is greater than apredefined ratio value, and whether the number of unique destinationinternet protocol addresses is greater than a predefined destinationinternet protocol address value.
 10. The non-transitorycomputer-readable medium of claim 9, wherein the using furthercomprises: when the ratio of the number of unique destination internetprotocol addresses and the total number of flows is not greater than thepredefined ratio value, and when the number of unique destinationinternet protocol addresses is not greater than the predefineddestination internet protocol address value, stored bin characteristicsof the one bin are erased.
 11. The non-transitory computer-readablemedium of claim 9, wherein the using further comprises: when the ratioof the number of unique destination internet protocol addresses and thetotal number of flows is greater than the predefined ratio value, andwhen the number of unique destination internet protocol addresses isgreater than the predefined destination internet protocol address value,a determination is made as to whether stored bin characteristics of theone bin match current bin characteristics of the one bin.
 12. Thenon-transitory computer-readable medium of claim 11, wherein the usingfurther comprises: when the determination that the stored bincharacteristics of the one bin does match the current bincharacteristics of the one bin, a flow count for the one bin is updated.13. The non-transitory computer-readable medium of claim 11, wherein theusing further comprises: when the determination that the stored bincharacteristics of the one bin does not match the current bincharacteristics of the one bin, stored bin characteristics of the onebin are erased.
 14. The non-transitory computer-readable medium of claim13, wherein the using further comprises: determining that flow dataassociated with the stored bin characteristics are associated with themultiport scan when the stored bin characteristics of the one bin isassociated with a flow count that exceeded a predefined threshold. 15.An apparatus for detecting a multiport scan, comprising: a processor;and a computer-readable medium storing instructions which, when executedby the processor, cause the processor to perform operations, theoperations comprising: allocating a plurality of flows into a pluralityof bins based upon a source internet protocol address of each of theplurality of flows, where each bin of the plurality of bins isassociated with a different source internet protocol address; generatinga set of bin characteristics for one bin of the plurality of bins whenthe one bin reaches a predefined flow capacity; separating a pluralityof flows that has been allocated into the one bin into a plurality ofcategories; determining a number of unique destination internet protocoladdresses and a total number of flows for each category of the pluralityof categories; and using the number of unique destination internetprotocol addresses and the total number of flows for each category ofthe plurality of categories to determine if the multiport scan exists.16. The apparatus of claim 15, wherein the using comprises: determiningwhether a ratio of the number of unique destination internet protocoladdresses and the total number of flows is greater than a predefinedratio value, and whether the number of unique destination internetprotocol addresses is greater than a predefined destination internetprotocol address value.
 17. The apparatus of claim 16, wherein the usingfurther comprises: when the ratio of the number of unique destinationinternet protocol addresses and the total number of flows is not greaterthan the predefined ratio value, and when the number of uniquedestination internet protocol addresses is not greater than thepredefined destination internet protocol address value, stored bincharacteristics of the one bin are erased.
 18. The apparatus of claim16, wherein the using further comprises: when the ratio of the number ofunique destination internet protocol addresses and the total number offlows is greater than the predefined ratio value, and when the number ofunique destination internet protocol addresses is greater than thepredefined destination internet protocol address value, a determinationis made as to whether stored bin characteristics of the one bin matchcurrent bin characteristics of the one bin.
 19. The apparatus of claim18, wherein the using further comprises: when the determination that thestored bin characteristics of the one bin does match the current bincharacteristics of the one bin, a flow count for the one bin is updated.20. The apparatus of claim 18, wherein the using further comprises: whenthe determination that the stored bin characteristics of the one bindoes not match the current bin characteristics of the one bin, storedbin characteristics of the one bin are erased; and determining that flowdata associated with the stored bin characteristics are associated withthe multiport scan when the stored bin characteristics of the one bin isassociated with a flow count that exceeded a predefined threshold.